Run started: 2026-05-13T01:40:10
Mode: FULL (multi-seed, repeated CV)

==============================================================================
SECTION 1: load & preprocess    [+0.0s since last banner]
==============================================================================
[Colab] Missing data files in /content/data: ['Breast_Cancer.csv', 'METABRIC_RNA_Mutation.csv']
[Colab] Opening upload dialog. Please upload: Breast_Cancer.csv, METABRIC_RNA_Mutation.csv
Saving Breast_Cancer.csv to Breast_Cancer.csv
Saving METABRIC_RNA_Mutation.csv to METABRIC_RNA_Mutation.csv
[Colab] Saved Breast_Cancer.csv -> /content/data/Breast_Cancer.csv
[Colab] Saved METABRIC_RNA_Mutation.csv -> /content/data/METABRIC_RNA_Mutation.csv
     Evaluation set    n  Alive (0)  Dead (1)  % Dead
       SEER - Train 2816       2385       431    15.3
         SEER - Val  604        511        93    15.4
        SEER - Test  604        512        92    15.2
 METABRIC - Imputed 1904        801      1103    57.9
METABRIC - Complete 1815        774      1041    57.4

==============================================================================
SECTION 4-5: NESTED CV (with per-fold strategy + alpha tuning)    [+98.1s since last banner]
==============================================================================
  [Repeat 1/5] Outer fold 1/5: train=2736 (420 dead) | val=684 (104 dead)
    inner strategy selection: 33.4s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'smote', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 32.7s  alpha(MLP+XGB)=0.05  alpha(MLP+LGBM)=0.30
    final fit+score: 3.8s   fold total: 69.9s
  [Repeat 1/5] Outer fold 2/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 33.8s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 37.9s  alpha(MLP+XGB)=0.95  alpha(MLP+LGBM)=0.30
    final fit+score: 4.3s   fold total: 76.0s
  [Repeat 1/5] Outer fold 3/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 31.2s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 31.6s  alpha(MLP+XGB)=0.80  alpha(MLP+LGBM)=0.75
    final fit+score: 5.5s   fold total: 68.3s
  [Repeat 1/5] Outer fold 4/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 36.0s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 37.1s  alpha(MLP+XGB)=0.35  alpha(MLP+LGBM)=0.20
    final fit+score: 8.6s   fold total: 81.8s
  [Repeat 1/5] Outer fold 5/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 34.5s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 36.1s  alpha(MLP+XGB)=0.30  alpha(MLP+LGBM)=0.30
    final fit+score: 6.7s   fold total: 77.4s
  [Repeat 2/5] Outer fold 1/5: train=2736 (420 dead) | val=684 (104 dead)
    inner strategy selection: 34.1s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 36.2s  alpha(MLP+XGB)=0.50  alpha(MLP+LGBM)=0.45
    final fit+score: 3.2s   fold total: 73.6s
  [Repeat 2/5] Outer fold 2/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 34.5s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 35.1s  alpha(MLP+XGB)=0.35  alpha(MLP+LGBM)=0.40
    final fit+score: 6.5s   fold total: 76.0s
  [Repeat 2/5] Outer fold 3/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 34.3s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 32.7s  alpha(MLP+XGB)=0.65  alpha(MLP+LGBM)=0.65
    final fit+score: 3.9s   fold total: 70.9s
  [Repeat 2/5] Outer fold 4/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 29.1s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 29.3s  alpha(MLP+XGB)=0.70  alpha(MLP+LGBM)=0.45
    final fit+score: 5.7s   fold total: 64.1s
  [Repeat 2/5] Outer fold 5/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 33.9s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'smote', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 36.9s  alpha(MLP+XGB)=0.05  alpha(MLP+LGBM)=0.10
    final fit+score: 4.9s   fold total: 75.7s
  [Repeat 3/5] Outer fold 1/5: train=2736 (420 dead) | val=684 (104 dead)
    inner strategy selection: 35.2s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'smote', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 39.4s  alpha(MLP+XGB)=0.05  alpha(MLP+LGBM)=0.30
    final fit+score: 4.5s   fold total: 79.1s
  [Repeat 3/5] Outer fold 2/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 33.3s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'smote', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 37.7s  alpha(MLP+XGB)=0.30  alpha(MLP+LGBM)=0.20
    final fit+score: 5.1s   fold total: 76.1s
  [Repeat 3/5] Outer fold 3/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 36.7s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 43.5s  alpha(MLP+XGB)=0.30  alpha(MLP+LGBM)=0.25
    final fit+score: 4.6s   fold total: 84.9s
  [Repeat 3/5] Outer fold 4/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 35.5s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 37.0s  alpha(MLP+XGB)=0.40  alpha(MLP+LGBM)=0.25
    final fit+score: 6.1s   fold total: 78.6s
  [Repeat 3/5] Outer fold 5/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 39.1s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 45.7s  alpha(MLP+XGB)=0.50  alpha(MLP+LGBM)=0.30
    final fit+score: 5.8s   fold total: 90.7s
  [Repeat 4/5] Outer fold 1/5: train=2736 (420 dead) | val=684 (104 dead)
    inner strategy selection: 32.7s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'none', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 31.3s  alpha(MLP+XGB)=0.40  alpha(MLP+LGBM)=0.45
    final fit+score: 4.7s   fold total: 68.7s
  [Repeat 4/5] Outer fold 2/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 32.7s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 34.3s  alpha(MLP+XGB)=0.70  alpha(MLP+LGBM)=0.45
    final fit+score: 7.0s   fold total: 74.0s
  [Repeat 4/5] Outer fold 3/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 34.1s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 34.3s  alpha(MLP+XGB)=0.30  alpha(MLP+LGBM)=0.40
    final fit+score: 4.6s   fold total: 72.9s
  [Repeat 4/5] Outer fold 4/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 35.5s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 37.1s  alpha(MLP+XGB)=0.35  alpha(MLP+LGBM)=0.20
    final fit+score: 5.7s   fold total: 78.3s
  [Repeat 4/5] Outer fold 5/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 38.2s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 41.6s  alpha(MLP+XGB)=0.40  alpha(MLP+LGBM)=0.35
    final fit+score: 6.8s   fold total: 86.6s
  [Repeat 5/5] Outer fold 1/5: train=2736 (420 dead) | val=684 (104 dead)
    inner strategy selection: 33.6s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 34.6s  alpha(MLP+XGB)=0.35  alpha(MLP+LGBM)=0.30
    final fit+score: 5.3s   fold total: 73.5s
  [Repeat 5/5] Outer fold 2/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 29.3s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 29.6s  alpha(MLP+XGB)=0.20  alpha(MLP+LGBM)=0.35
    final fit+score: 4.9s   fold total: 63.8s
  [Repeat 5/5] Outer fold 3/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 32.2s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 31.8s  alpha(MLP+XGB)=0.20  alpha(MLP+LGBM)=0.20
    final fit+score: 5.8s   fold total: 69.9s
  [Repeat 5/5] Outer fold 4/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 31.7s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 27.5s  alpha(MLP+XGB)=0.35  alpha(MLP+LGBM)=0.65
    final fit+score: 4.7s   fold total: 63.9s
  [Repeat 5/5] Outer fold 5/5: train=2736 (419 dead) | val=684 (105 dead)
    inner strategy selection: 34.0s  picks={'LR': 'weights', 'RF': 'weights', 'XGB': 'weights', 'LGBM': 'weights', 'MLP': 'smote'}
    inner alpha tuning: 36.8s  alpha(MLP+XGB)=0.60  alpha(MLP+LGBM)=0.40
    final fit+score: 6.3s   fold total: 77.1s

Nested CV summary (Table 11 replacement):
   model  f1_macro_mean  f1_macro_std  f1_dead_mean  f1_dead_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  recall_dead_mean  brier_mean
      LR       0.616381      0.014559      0.414022     0.021162  0.749495 0.020252     0.411705    0.043118          0.638168    0.196660
      RF       0.619727      0.026260      0.332933     0.046850  0.707211 0.023243     0.361553    0.045692          0.267520    0.131757
     XGB       0.591599      0.017806      0.302325     0.032090  0.660517 0.020795     0.306010    0.038813          0.288835    0.153186
    LGBM       0.591412      0.019935      0.326264     0.033076  0.664739 0.024425     0.322688    0.036976          0.374022    0.167225
     MLP       0.587498      0.023415      0.298318     0.040307  0.634385 0.034782     0.304682    0.044853          0.291535    0.167293
 MLP+XGB       0.598093      0.021811      0.306679     0.039454  0.663970 0.022508     0.316997    0.040638          0.276249    0.146147
MLP+LGBM       0.603201      0.018836      0.328278     0.033449  0.670707 0.023803     0.330353    0.040879          0.329733    0.151285

==============================================================================
SECTION 11: pairwise paired tests (corrected + Holm-Bonferroni)    [+1871.8s since last banner]
==============================================================================

Pairwise tests (first 20 rows):
model_a  model_b   metric  k_folds  mean_diff   t_naive      p_naive  t_nb_corrected  p_nb_corrected  p_nb_holm_adj  holm_reject_at_005
     LR       RF f1_macro       25  -0.003346 -0.673578 5.070153e-01       -0.250161        0.804592       1.000000               False
     LR      XGB f1_macro       25   0.024782  6.280815 1.715174e-06        2.332636        0.028380       0.454080               False
     LR     LGBM f1_macro       25   0.024969  6.382572 1.340309e-06        2.370428        0.026145       0.444463               False
     LR      MLP f1_macro       25   0.028883  6.076754 2.822112e-06        2.256850        0.033397       0.500961               False
     LR  MLP+XGB f1_macro       25   0.018288  4.045156 4.700596e-04        1.502333        0.146056       1.000000               False
     LR MLP+LGBM f1_macro       25   0.013180  3.143169 4.405696e-03        1.167343        0.254539       1.000000               False
     RF      XGB f1_macro       25   0.028128  6.649696 7.055149e-07        2.469635        0.021025       0.378453               False
     RF     LGBM f1_macro       25   0.028315  7.029957 2.872228e-07        2.610860        0.015323       0.306465               False
     RF      MLP f1_macro       25   0.032229  9.159186 2.654672e-09        3.401636        0.002348       0.049311                True
     RF  MLP+XGB f1_macro       25   0.021634  6.875756 4.126169e-07        2.553592        0.017435       0.331267               False
     RF MLP+LGBM f1_macro       25   0.016526  4.998669 4.170898e-05        1.856459        0.075706       1.000000               False
    XGB     LGBM f1_macro       25   0.000187  0.060399 9.523380e-01        0.022432        0.982289       1.000000               False
    XGB      MLP f1_macro       25   0.004101  0.999500 3.275240e-01        0.371205        0.713742       1.000000               False
    XGB  MLP+XGB f1_macro       25  -0.006494 -2.091693 4.722399e-02       -0.776835        0.444846       1.000000               False
    XGB MLP+LGBM f1_macro       25  -0.011602 -3.096155 4.932810e-03       -1.149883        0.261522       1.000000               False
   LGBM      MLP f1_macro       25   0.003914  0.824994 4.174997e-01        0.306395        0.761946       1.000000               False
   LGBM  MLP+XGB f1_macro       25  -0.006682 -1.597400 1.232609e-01       -0.593260        0.558557       1.000000               False
   LGBM MLP+LGBM f1_macro       25  -0.011789 -3.531391 1.704598e-03       -1.311526        0.202089       1.000000               False
    MLP  MLP+XGB f1_macro       25  -0.010595 -3.097985 4.911213e-03       -1.150563        0.261247       1.000000               False
    MLP MLP+LGBM f1_macro       25  -0.015703 -4.409516 1.864435e-04       -1.637653        0.114540       1.000000               False

==============================================================================
SECTION 5b: ABLATION A (per-model imbalance handling)    [+0.1s since last banner]
==============================================================================
model      strategy  f1_dead_mean  f1_dead_std  f1_macro_mean  f1_macro_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  recall_dead_mean  brier_mean
  MLP          none      0.236699     0.065759       0.578344      0.032881  0.736574 0.011765     0.395895    0.030663          0.150806    0.114719
  MLP         smote      0.301068     0.036501       0.590079      0.018387  0.632768 0.016413     0.310801    0.035982          0.291886    0.165830
  XGB          none      0.270898     0.049311       0.588113      0.026969  0.675617 0.017856     0.314882    0.029737          0.204121    0.133223
  XGB         smote      0.295497     0.028215       0.600393      0.015263  0.676892 0.020257     0.328925    0.029218          0.228956    0.132317
  XGB       weights      0.294975     0.030984       0.584691      0.016809  0.665885 0.023793     0.308120    0.036348          0.291886    0.156040
  XGB smote_weights      0.295497     0.028215       0.600393      0.015263  0.676892 0.020257     0.328925    0.029218          0.228956    0.132317
 LGBM          none      0.283394     0.032908       0.596578      0.016692  0.681187 0.018821     0.339712    0.018380          0.208004    0.128411
 LGBM         smote      0.290541     0.038177       0.598976      0.021016  0.672245 0.014216     0.332461    0.027187          0.219451    0.131299
 LGBM       weights      0.332045     0.020929       0.597438      0.010166  0.676525 0.015953     0.335477    0.025910          0.370183    0.163428
 LGBM smote_weights      0.290541     0.038177       0.598976      0.021016  0.672245 0.014216     0.332461    0.027187          0.219451    0.131299

==============================================================================
SECTION 6: ABLATION B (architecture with own best strategy)    [+35.5s since last banner]
==============================================================================
  Ablation B best-per-model strategies:
    LR     -> weights
    RF     -> weights
    MLP    -> smote
    XGB    -> smote
    LGBM   -> weights
   model strategy  f1_dead_mean  f1_dead_std  f1_macro_mean  f1_macro_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  recall_dead_mean  brier_mean
      LR  weights      0.415660     0.021209       0.617213      0.016339  0.750855 0.008579     0.412406    0.039116          0.641190    0.196072
      RF  weights      0.324900     0.033556       0.616145      0.019200  0.704301 0.012846     0.351850    0.035917          0.255659    0.131662
     XGB    smote      0.295497     0.031545       0.600393      0.017065  0.676892 0.022648     0.328925    0.032667          0.228956    0.132317
    LGBM  weights      0.332045     0.023400       0.597438      0.011366  0.676525 0.017836     0.335477    0.028968          0.370183    0.163428
     MLP    smote      0.301068     0.040809       0.590079      0.020558  0.632768 0.018351     0.310801    0.040229          0.291886    0.165830
 MLP+XGB   hybrid      0.291800     0.034246       0.595542      0.017346  0.666985 0.010651     0.335210    0.040633          0.238480    0.136972
MLP+LGBM   hybrid      0.326144     0.023446       0.601998      0.014752  0.677676 0.011056     0.345132    0.030371          0.326337    0.149403

==============================================================================
SECTION 7: MULTI-SEED HELD-OUT TEST + METABRIC    [+209.3s since last banner]
==============================================================================

  Seed 42:
    Tuned alphas (full features):  MLP+XGB=0.10  MLP+LGBM=0.50
    Tuned alphas (shared 6):       MLP+XGB=0.75  MLP+LGBM=0.80

  Seed 123:
    Tuned alphas (full features):  MLP+XGB=0.25  MLP+LGBM=0.35
    Tuned alphas (shared 6):       MLP+XGB=0.80  MLP+LGBM=0.85

  Seed 456:
    Tuned alphas (full features):  MLP+XGB=0.15  MLP+LGBM=0.40
    Tuned alphas (shared 6):       MLP+XGB=0.45  MLP+LGBM=0.55

  Seed 789:
    Tuned alphas (full features):  MLP+XGB=0.30  MLP+LGBM=0.40
    Tuned alphas (shared 6):       MLP+XGB=0.35  MLP+LGBM=0.80

  Seed 2024:
    Tuned alphas (full features):  MLP+XGB=0.40  MLP+LGBM=0.40
    Tuned alphas (shared 6):       MLP+XGB=0.60  MLP+LGBM=0.90

SEER test (multi-seed, Dead = positive class):
 eval_set    model  n_seeds  accuracy_mean  accuracy_std  f1_macro_mean  f1_macro_std  f1_dead_mean  f1_dead_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  precision_dead_mean  recall_dead_mean  brier_mean
SEER_test     LGBM        5       0.758609      0.005022       0.595615      0.003275      0.338911     0.008367  0.664644 0.006144     0.255346    0.006910             0.290832          0.406522    0.173476
SEER_test       LR        5       0.685430      0.000000       0.582412      0.000000      0.375000     0.000000  0.723803 0.000000     0.345240    0.000000             0.268868          0.619565    0.205080
SEER_test      MLP        5       0.796358      0.023179       0.571466      0.022583      0.261241     0.038741  0.635988 0.024885     0.275122    0.026535             0.297300          0.236957    0.162532
SEER_test MLP+LGBM        5       0.800662      0.011387       0.593978      0.013290      0.304364     0.024695  0.675272 0.008451     0.269672    0.013667             0.326249          0.286957    0.150086
SEER_test  MLP+XGB        5       0.821854      0.008328       0.553597      0.014157      0.207601     0.029867  0.689691 0.007254     0.264049    0.008880             0.323959          0.154348    0.135277
SEER_test       RF        5       0.820530      0.005923       0.580758      0.008888      0.263710     0.014471  0.699083 0.004449     0.294329    0.003285             0.352233          0.210870    0.136618
SEER_test      XGB        5       0.822185      0.003812       0.549770      0.007238      0.199564     0.013935  0.681501 0.006510     0.257461    0.004860             0.317874          0.145652    0.137052

METABRIC imputed (multi-seed):
        eval_set    model  n_seeds  accuracy_mean  accuracy_std  f1_macro_mean  f1_macro_std  f1_dead_mean  f1_dead_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  precision_dead_mean  recall_dead_mean  brier_mean
METABRIC_imputed     LGBM        5       0.569538      0.006439       0.569496      0.006482      0.569456     0.010383  0.638861 0.001389     0.681169    0.003701             0.676888          0.491568    0.270673
METABRIC_imputed       LR        5       0.595063      0.000000       0.589931      0.000000      0.635805     0.000000  0.625847 0.000000     0.682745    0.000000             0.663708          0.610154    0.242792
METABRIC_imputed      MLP        5       0.500105      0.014585       0.472056      0.027949      0.354392     0.067485  0.608454 0.026567     0.672613    0.013192             0.704335          0.241704    0.343181
METABRIC_imputed MLP+LGBM        5       0.502311      0.015387       0.477962      0.027586      0.368994     0.064672  0.628796 0.018632     0.680441    0.010214             0.694690          0.255666    0.314395
METABRIC_imputed  MLP+XGB        5       0.495693      0.011846       0.469136      0.023877      0.353188     0.057049  0.592694 0.007880     0.656646    0.007911             0.688668          0.240979    0.334698
METABRIC_imputed       RF        5       0.491282      0.001686       0.461870      0.002220      0.336093     0.005168  0.626807 0.001626     0.674653    0.001093             0.688913          0.222303    0.304523
METABRIC_imputed      XGB        5       0.482038      0.003776       0.467401      0.006142      0.379551     0.016192  0.562420 0.004677     0.627240    0.004065             0.620203          0.273799    0.351622

==============================================================================
SECTION 8: BOOTSTRAP CIs (M1, M9) on first-seed predictions    [+522.1s since last banner]
==============================================================================

Pointwise bootstrap CIs (head):
model   metric    point  boot_mean    ci_lo    ci_hi         eval_set
   LR f1_macro 0.589931   0.590276 0.568037 0.610947 METABRIC_imputed
   LR  f1_dead 0.635805   0.635880 0.611887 0.658207 METABRIC_imputed
   LR      auc 0.625847   0.626619 0.601172 0.652288 METABRIC_imputed
   LR   pr_auc 0.682745   0.684056 0.653136 0.715555 METABRIC_imputed
   RF f1_macro 0.461805   0.462458 0.441665 0.482485 METABRIC_imputed
   RF  f1_dead 0.337432   0.338502 0.307779 0.368246 METABRIC_imputed
   RF      auc 0.627873   0.627935 0.603320 0.653783 METABRIC_imputed
   RF   pr_auc 0.675853   0.676647 0.644555 0.706623 METABRIC_imputed
  XGB f1_macro 0.467595   0.468052 0.445050 0.490981 METABRIC_imputed
  XGB  f1_dead 0.381191   0.381686 0.351662 0.412865 METABRIC_imputed
  XGB      auc 0.557742   0.558055 0.530985 0.584414 METABRIC_imputed
  XGB   pr_auc 0.624732   0.625761 0.592785 0.657039 METABRIC_imputed
 LGBM f1_macro 0.571427   0.571699 0.550765 0.593481 METABRIC_imputed
 LGBM  f1_dead 0.570526   0.570856 0.544673 0.597094 METABRIC_imputed
 LGBM      auc 0.641025   0.641175 0.617540 0.666804 METABRIC_imputed

Pairwise bootstrap CIs vs LR (head):
ref_model model   metric  mean_diff     ci_lo     ci_hi  crosses_zero         eval_set
       LR    RF f1_macro  -0.127818 -0.153051 -0.102328         False METABRIC_imputed
       LR    RF  f1_dead  -0.297378 -0.329760 -0.265293         False METABRIC_imputed
       LR    RF      auc   0.001316 -0.020454  0.022600          True METABRIC_imputed
       LR    RF   pr_auc  -0.007408 -0.027624  0.011840          True METABRIC_imputed
       LR   XGB f1_macro  -0.122224 -0.150018 -0.094954         False METABRIC_imputed
       LR   XGB  f1_dead  -0.254194 -0.287561 -0.220472         False METABRIC_imputed
       LR   XGB      auc  -0.068564 -0.095797 -0.042184         False METABRIC_imputed
       LR   XGB   pr_auc  -0.058295 -0.081261 -0.035939         False METABRIC_imputed
       LR  LGBM f1_macro  -0.018578 -0.041697  0.004676          True METABRIC_imputed
       LR  LGBM  f1_dead  -0.065024 -0.093190 -0.039152         False METABRIC_imputed
       LR  LGBM      auc   0.014556 -0.007846  0.038107          True METABRIC_imputed
       LR  LGBM   pr_auc  -0.000422 -0.022009  0.018871          True METABRIC_imputed
       LR   MLP f1_macro  -0.138693 -0.164864 -0.113091         False METABRIC_imputed
       LR   MLP  f1_dead  -0.327154 -0.361052 -0.294177         False METABRIC_imputed
       LR   MLP      auc  -0.043358 -0.067326 -0.019896         False METABRIC_imputed

==============================================================================
SECTION 9: CALIBRATION (M10) — reliability + Brier    [+318.9s since last banner]
==============================================================================
        eval_set    model    brier  ece_uniform
       SEER_test       LR 0.205080     0.282419
       SEER_test       RF 0.136311     0.105447
       SEER_test      XGB 0.137166     0.080810
       SEER_test     LGBM 0.171447     0.155437
       SEER_test      MLP 0.165363     0.151229
       SEER_test  MLP+XGB 0.135481     0.071527
       SEER_test MLP+LGBM 0.148864     0.107597
METABRIC_imputed       LR 0.242792     0.120448
METABRIC_imputed       RF 0.304066     0.253521
METABRIC_imputed      XGB 0.353321     0.301268
METABRIC_imputed     LGBM 0.269224     0.187606
METABRIC_imputed      MLP 0.384720     0.360834
METABRIC_imputed  MLP+XGB 0.363195     0.332871
METABRIC_imputed MLP+LGBM 0.346109     0.320347

==============================================================================
SECTION 10b: FIGURES    [+0.4s since last banner]
==============================================================================

==============================================================================
DONE    [+2.2s since last banner]
==============================================================================
Run completed: 2026-05-13T02:31:09
All results in: /content/outputs
Figures in:     /content/outputs/figures
Run log:        /content/outputs/run_log.txt
    7.8 KB  outputs/ablation_b_fold_scores.csv
    4.3 KB  outputs/best_strategy_per_fold.csv
    7.0 KB  outputs/bootstrap_cis_pairwise.csv
    8.5 KB  outputs/bootstrap_cis_pointwise.csv
    0.8 KB  outputs/calibration_metrics.csv
    0.9 KB  outputs/hybrid_alphas_per_fold.csv
   25.3 KB  outputs/multiseed_per_seed_scores.csv
   39.3 KB  outputs/nested_cv_fold_scores.csv
    8.7 KB  outputs/pairwise_tests_corrected.csv
   25.3 KB  outputs/run_log.txt
    2.0 KB  outputs/table10_seer_test_multiseed.csv
    1.5 KB  outputs/table11_seer_cv_nested.csv
    2.0 KB  outputs/table12_metabric_multiseed.csv
    2.0 KB  outputs/table12b_metabric_complete_multiseed.csv
    2.2 KB  outputs/table13_ablation_a.csv
    1.6 KB  outputs/table14_ablation_b.csv
    0.2 KB  outputs/table9_partitions.csv
   79.3 KB  outputs/figures/fig_ablation_a.png
  114.5 KB  outputs/figures/fig_ablation_b.png
  118.1 KB  outputs/figures/fig_confusion_matrices.png
   99.6 KB  outputs/figures/fig_cross_cohort_auc.png
  195.2 KB  outputs/figures/fig_pr_metabric.png
  179.2 KB  outputs/figures/fig_pr_seer.png
  295.8 KB  outputs/figures/fig_reliability.png
  169.6 KB  outputs/figures/fig_roc_metabric.png
  141.5 KB  outputs/figures/fig_roc_seer.png
=== Table 11 (nested CV — SEER) ===
   model  f1_macro_mean  f1_macro_std  f1_dead_mean  f1_dead_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  recall_dead_mean  brier_mean
      LR       0.616381      0.014559      0.414022     0.021162  0.749495 0.020252     0.411705    0.043118          0.638168    0.196660
      RF       0.619727      0.026260      0.332933     0.046850  0.707211 0.023243     0.361553    0.045692          0.267520    0.131757
     XGB       0.591599      0.017806      0.302325     0.032090  0.660517 0.020795     0.306010    0.038813          0.288835    0.153186
    LGBM       0.591412      0.019935      0.326264     0.033076  0.664739 0.024425     0.322688    0.036976          0.374022    0.167225
     MLP       0.587498      0.023415      0.298318     0.040307  0.634385 0.034782     0.304682    0.044853          0.291535    0.167293
 MLP+XGB       0.598093      0.021811      0.306679     0.039454  0.663970 0.022508     0.316997    0.040638          0.276249    0.146147
MLP+LGBM       0.603201      0.018836      0.328278     0.033449  0.670707 0.023803     0.330353    0.040879          0.329733    0.151285

=== Table 10 (SEER test, multi-seed) ===
 eval_set    model  n_seeds  accuracy_mean  accuracy_std  f1_macro_mean  f1_macro_std  f1_dead_mean  f1_dead_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  precision_dead_mean  recall_dead_mean  brier_mean
SEER_test     LGBM        5       0.758609      0.005022       0.595615      0.003275      0.338911     0.008367  0.664644 0.006144     0.255346    0.006910             0.290832          0.406522    0.173476
SEER_test       LR        5       0.685430      0.000000       0.582412      0.000000      0.375000     0.000000  0.723803 0.000000     0.345240    0.000000             0.268868          0.619565    0.205080
SEER_test      MLP        5       0.796358      0.023179       0.571466      0.022583      0.261241     0.038741  0.635988 0.024885     0.275122    0.026535             0.297300          0.236957    0.162532
SEER_test MLP+LGBM        5       0.800662      0.011387       0.593978      0.013290      0.304364     0.024695  0.675272 0.008451     0.269672    0.013667             0.326249          0.286957    0.150086
SEER_test  MLP+XGB        5       0.821854      0.008328       0.553597      0.014157      0.207601     0.029867  0.689691 0.007254     0.264049    0.008880             0.323959          0.154348    0.135277
SEER_test       RF        5       0.820530      0.005923       0.580758      0.008888      0.263710     0.014471  0.699083 0.004449     0.294329    0.003285             0.352233          0.210870    0.136618
SEER_test      XGB        5       0.822185      0.003812       0.549770      0.007238      0.199564     0.013935  0.681501 0.006510     0.257461    0.004860             0.317874          0.145652    0.137052

=== Table 12 (METABRIC imputed, multi-seed) ===
        eval_set    model  n_seeds  accuracy_mean  accuracy_std  f1_macro_mean  f1_macro_std  f1_dead_mean  f1_dead_std  auc_mean  auc_std  pr_auc_mean  pr_auc_std  precision_dead_mean  recall_dead_mean  brier_mean
METABRIC_imputed     LGBM        5       0.569538      0.006439       0.569496      0.006482      0.569456     0.010383  0.638861 0.001389     0.681169    0.003701             0.676888          0.491568    0.270673
METABRIC_imputed       LR        5       0.595063      0.000000       0.589931      0.000000      0.635805     0.000000  0.625847 0.000000     0.682745    0.000000             0.663708          0.610154    0.242792
METABRIC_imputed      MLP        5       0.500105      0.014585       0.472056      0.027949      0.354392     0.067485  0.608454 0.026567     0.672613    0.013192             0.704335          0.241704    0.343181
METABRIC_imputed MLP+LGBM        5       0.502311      0.015387       0.477962      0.027586      0.368994     0.064672  0.628796 0.018632     0.680441    0.010214             0.694690          0.255666    0.314395
METABRIC_imputed  MLP+XGB        5       0.495693      0.011846       0.469136      0.023877      0.353188     0.057049  0.592694 0.007880     0.656646    0.007911             0.688668          0.240979    0.334698
METABRIC_imputed       RF        5       0.491282      0.001686       0.461870      0.002220      0.336093     0.005168  0.626807 0.001626     0.674653    0.001093             0.688913          0.222303    0.304523
METABRIC_imputed      XGB        5       0.482038      0.003776       0.467401      0.006142      0.379551     0.016192  0.562420 0.004677     0.627240    0.004065             0.620203          0.273799    0.351622

=== Calibration (M10) ===
        eval_set    model    brier  ece_uniform
       SEER_test       LR 0.205080     0.282419
       SEER_test       RF 0.136311     0.105447
       SEER_test      XGB 0.137166     0.080810
       SEER_test     LGBM 0.171447     0.155437
       SEER_test      MLP 0.165363     0.151229
       SEER_test  MLP+XGB 0.135481     0.071527
       SEER_test MLP+LGBM 0.148864     0.107597
METABRIC_imputed       LR 0.242792     0.120448
METABRIC_imputed       RF 0.304066     0.253521
METABRIC_imputed      XGB 0.353321     0.301268
METABRIC_imputed     LGBM 0.269224     0.187606
METABRIC_imputed      MLP 0.384720     0.360834
METABRIC_imputed  MLP+XGB 0.363195     0.332871
METABRIC_imputed MLP+LGBM 0.346109     0.320347

--- fig_ablation_a.png ---

--- fig_ablation_b.png ---

--- fig_confusion_matrices.png ---

--- fig_cross_cohort_auc.png ---

--- fig_pr_metabric.png ---

--- fig_pr_seer.png ---

--- fig_reliability.png ---

--- fig_roc_metabric.png ---

--- fig_roc_seer.png ---
  adding: outputs/ (stored 0%)
  adding: outputs/table10_seer_test_multiseed.csv (deflated 52%)
  adding: outputs/ablation_b_fold_scores.csv (deflated 57%)
  adding: outputs/table12_metabric_multiseed.csv (deflated 51%)
  adding: outputs/run_log.txt (deflated 81%)
  adding: outputs/figures/ (stored 0%)
  adding: outputs/figures/fig_pr_metabric.png (deflated 5%)
  adding: outputs/figures/fig_roc_seer.png (deflated 10%)
  adding: outputs/figures/fig_confusion_matrices.png (deflated 23%)
  adding: outputs/figures/fig_roc_metabric.png (deflated 7%)
  adding: outputs/figures/fig_ablation_b.png (deflated 21%)
  adding: outputs/figures/fig_cross_cohort_auc.png (deflated 18%)
  adding: outputs/figures/fig_pr_seer.png (deflated 6%)
  adding: outputs/figures/fig_ablation_a.png (deflated 18%)
  adding: outputs/figures/fig_reliability.png (deflated 4%)
  adding: outputs/table14_ablation_b.csv (deflated 49%)
  adding: outputs/table13_ablation_a.csv (deflated 58%)
  adding: outputs/multiseed_per_seed_scores.csv (deflated 62%)
  adding: outputs/best_strategy_per_fold.csv (deflated 63%)
  adding: outputs/pairwise_tests_corrected.csv (deflated 56%)
  adding: outputs/table9_partitions.csv (deflated 26%)
  adding: outputs/bootstrap_cis_pairwise.csv (deflated 63%)
  adding: outputs/bootstrap_cis_pointwise.csv (deflated 59%)
  adding: outputs/table11_seer_cv_nested.csv (deflated 49%)
  adding: outputs/table12b_metabric_complete_multiseed.csv (deflated 51%)
  adding: outputs/calibration_metrics.csv (deflated 51%)
  adding: outputs/nested_cv_fold_scores.csv (deflated 62%)
  adding: outputs/hybrid_alphas_per_fold.csv (deflated 75%)
